Advancing Distributed Data Management for the HydroShare Hydrologic Information System

HydroShare (https://www.hydroshare.org) is an online collaborative system to support the open sharing of hydrologic data, analytical tools, and computer models. Hydrologic data and models are often large, extending to multi-gigabyte or terabyte scale, and as a result, the scalability of centralized data management poses challenges for a system such as HydroShare. A distributed data management framework that enables distributed physical data storage and management in multiple locations thus becomes a necessity. We use the iRODS (Integrated Rule-Oriented Data System) data grid middleware as the distributed data storage and management back end in HydroShare. iRODS provides a unified virtual file system for distributed physical storages in multiple locations and enables data federation across geographically dispersed institutions around the world. In this paper, we describe the iRODS-based distributed data management approaches implemented in HydroShare to provide a practical demonstration of a production system for supporting big data in the environmental sciences

High Performance Computing Instrumentation and Research Productivity in U.S. Universities

This paper studies the relationship between investments in High-Performance Computing (HPC) instrumentation and research competitiveness. Measures of institutional HPC investment are computed from data that is readily available from the Top 500 list, a list that has been published twice a year since 1993 that lists the fastest 500 computers in the world at that time. Institutions that are studied include US doctoral-granting institutions that fall into the very high or high research rankings according to the Carnegie Foundation classifications and additional institutions that have had entries in the Top 500 list. Research competitiveness is derived from federal funding data, compilations of scholarly publications, and institutional rankings. Correlation and Two Stage Least Square regression is used to analyze the research-related returns to investment in HPC. Two models are examined and give results that are both economically and statistically significant. Appearance on the Top 500 list is associated with a contemporaneous increase in NSF funding levels as well as a contemporaneous increase in the number of publications. The rate of depreciation in returns to HPC is rapid. The conclusion is that consistent investments in HPC at even modest levels are strongly correlated to research competitiveness

HydroShare – A Case Study of the Application of Modern Software Engineering to a Large Distributed Federally-Funded Scientific Software Development Project

HydroShare is an online collaborative system under development to support the open sharing of hydrologic data, analytical tools, and computer models. With HydroShare, scientists can easily discover, access, and analyze hydrologic data and thereby enhance the production and reproducibility of hydrologic scientific results. HydroShare also takes advantage of emerging social media functionality to enable users to enhance information about and collaboration around hydrologic data and models. HydroShare is being developed by an interdisciplinary collaborative team of domain scientists, university software developers, and professional software engineers from ten institutions located across the United States. While the combination of non–co-located, diverse stakeholders presents communication and management challenges, the interdisciplinary nature of the team is integral to the project’s goal of improving scientific software development and capabilities in academia. This chapter describes the challenges faced and lessons learned with the development of HydroShare, as well as the approach to software development that the HydroShare team adopted on the basis of the lessons learned. The chapter closes with recommendations for the application of modern software engineering techniques to large, collaborative, scientific software development projects, similar to the National Science Foundation (NSF)–funded HydroShare, in order to promote the successful application of the approach described herein by other teams for other projects

Autonomous Robotic Sensing Experiments at San Joaquin River

Distributed, high-density spatiotemporal observations are proposed for answering many river-related questions, including those pertaining to hydraulics and multi-dimensional river modeling, geomorphology, sediment transport and riparian habitat restoration. We present here a case study of an autonomous, high-resolution robotic spatial mapping of cross-sectional velocity and salt concentration in a river basin. Several experiments for analyzing the spatial and temporal trends at multiple cross-sections of the San Joaquin River were performed during the campaign from August 21-25, 2006. Preliminary analysis from these experiments illustrating the range of investigations is presented. Lessons learned during the campaign are discussed to provide useful insights for similar robotic investigations in aquatic environments

HydroShare: Sharing Diverse Environmental Data Types and Models as Social Objects with Application to the Hydrology Domain

The types of data and models used within the hydrologic science community are diverse. New repositories have succeeded in making data and models more accessible, but are, in most cases, limited to particular types or classes of data or models and also lack the type of collaborative and iterative functionality needed to enable shared data collection and modeling workflows. File sharing systems currently used within many scientific communities for private sharing of preliminary and intermediate data and modeling products do not support collaborative data capture, description, visualization, and annotation. In this article, we cast hydrologic datasets and models as “social objects” that can be published, collaborated around, annotated, discovered, and accessed. This article describes the generic data model and content packaging scheme for diverse hydrologic datasets and models used by a new hydrologic collaborative environment called HydroShare to enable storage, management, sharing, publication, and annotation of the diverse types of data and models used by hydrologic scientists. The flexibility of HydroShare\u27s data model and packaging scheme is demonstrated using multiple hydrologic data and model use cases that highlight its features

Nims3d: A novel rapidly deployable robot for 3-dimensional applications

Abstract — In this paper, we present NIMS3D, a novel, rapidly deployable cable based robotic system capable of accurate positioning within its 3-dimensional span. The system is designed for indoor and outdoor use. In NIMS3D, a node moves via three cables which enable navigation in the 3D volume spanned by the system. The hardware is composed primarily of commercially available components and the software consists of three tiers: low level motor control, motion planning, and user interface. The proposed system has health monitoring capabilities that seek to ensure that robot integrity is not compromised. We provide theoretical and empirical analysis of system characteristics and present results that advocate its use for a variety of applications such as topographical and optical intensity mapping. Finally, we propose a number of future enhancements and plans for the system. I

Temporal and spatial scales of temperature change in an alpine fell field ecosystem

Cool ambient air temperatures and warm soil and rock surfaces in alpine fellfield ecosystems produce sharp and dynamic spatial gradients in surface temperatures, with profound impacts on plant ecophysiology. Spot measurements of plant and soil surface temperatures are not sufficient to understand the complexity of this environment. Thermal scanner instrumentation mounted on a NIMS-RD system can provide dense measurements of spatial and temporal patterns of surface temperatures across a fellfield habitat, as well as a photographic log of transect position

MAS 1: Multiscale Sensing and Actuation Architecture and Performance

Many environmental applications require high fidelity sampling of temporally and spatially distributed phenomena. We propose a MultiScale Sampling approach for efficiently sampling such phenomena. This approach introduces a hierarchy of sensors according to the sampling fidelity, spatial coverage, and mobility characteristics. We report the development of a two-tier MultiScale System where information from a low fidelity, high spatial (global) sensor actuates a mobile robotic node, carrying a high fidelity, low spatial coverage (spot measurement) sensor, to perform guided sampling in regions of high phenomenon variability. As a case study of the proposed MultiScale paradigm, we investigated the spatiotemporal distribution of light intensity in a forest under story. Performance of the MultiScale approach is verified in simulation and on a physical system. We also compare the two most recently used sampling approaches, Adaptive Sampling and MultiScale Sampling, empirically in simulation for both static and spatiotemporally varying phenomenon. Results indicate that MultiScale Sampling is suitable for the sampling of high spatiotemporal frequency varying fields and significantly outperforms Adaptive Sampling